Sequential Reinstatement of Neocortical Activity during Slow Oscillations Depends on Cells’ Global Activity

During Slow Wave Sleep (SWS), cortical activity is dominated by endogenous processes modulated by slow oscillations (0.1–1 Hz): cell ensembles fluctuate between states of sustained activity (UP states) and silent epochs (DOWN states). We investigate here the temporal structure of ensemble activity during UP states by means of multiple single unit recordings in the prefrontal cortex of naturally sleeping rats. As previously shown, the firing rate of each PFC cell peaks at a distinct time lag after the DOWN/UP transition in a consistent order. We show here that, conversely, the latency of the first spike after the UP state onset depends primarily on the session-averaged firing rates of cells (which can be considered as an indirect measure of their intrinsic excitability). This latency can be explained by a simple homogeneous process (Poisson model) of cell firing, with sleep averaged firing rates employed as parameters. Thus, at DOWN/UP transitions, neurons are affected both by a slow process, possibly originating in the cortical network, modulating the time course of firing for each cell, and by a fast, relatively stereotyped reinstatement of activity, related mostly to global activity levels

Design of a biologically inspired navigation system for the Psikharpax rodent robot

This work presents the development and implementation of a biologically inspired navigation system on the autonomous Psikharpax rodent robot. Our system comprises two independent navigation strategies: a taxon expert and a planning expert. The presented navigation system allows the robot to learn the optimal strategy in each situation, by relying upon a strategy selection mechanism

Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning

International audienceCurrent learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus–response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal

Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia

We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This “exploration-exploitation” trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine’s involvement in the exploration-exploitation trade-off with the existing evidence for basal ganglia control of action selection, by testing the hypothesis that tonic dopamine in the striatum, the basal ganglia’s input nucleus, sets the current exploration-exploitation trade-off. We first advance the idea of interpreting the basal ganglia output as a probability distribution function for action selection. Using computational models of the full basal ganglia circuit, we showed that, under this interpretation, the actions of dopamine within the striatum change the basal ganglia’s output to favor the level of exploration or exploitation encoded in the probability distribution. We also found that our models predict striatal dopamine controls the exploration-exploitation trade-off if we instead read-out the probability distribution from the target nuclei of the basal ganglia, where their inhibitory input shapes the cortical input to these nuclei. Finally, by integrating the basal ganglia within a reinforcement learning model, we showed how dopamine’s effect on the exploration-exploitation trade-off could be measurable in a forced two-choice task. These simulations also showed how tonic dopamine can appear to affect learning while only directly altering the trade-off. Thus, our models support the hypothesis that changes in tonic dopamine within the striatum can alter the exploration-exploitation trade-off by modulating the output of the basal ganglia